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RESPONSE TO AMENDMENT 

1 . Applicants' amendment filed, 8/19/2008, see page 6 through page 12, with respect to the 
rejection of claims 1-20 have been fully considered and but they are not persuasive. 

Applicant argues in essence regarding claim 1, 5, 6 and 7 that Rhoads fails to teach "video 
signal to locate image areas in which the video signal changes from frame to frame". 

Contrary to the applicant's assertion, limitations in the "amended claim", the Examiner 
would like to point out that, Rhoads discloses (column 1, lines 41 through column 2, line 2, 
a media signal, such as an image (video) signal, is modified such that the embedded code 
(watermark) is imperceptible or nearly imperceptible to the eye, yet may be detected (located) 
through an automated detection process, the reading component analyses content (examines) 
content to detect whether a watermark is present, video objects in a video sequence (frame-to- 
frame) with object specific actions or information (watermark), a video object refers to a spatial 
or temporal portion of a video signal. The embedding component embeds a watermark by 
altering data samples of the media content. The reading component analyzes content to detect 
whether a watermark is present. The invention provides methods and systems for associating 
video objects in a "video sequence" with object specific actions or information using auxiliary 
information embedded in video frames or audio tracks. A video object refers to a spatial and 
temporal portion of a video signal that depicts a recognizable object, such as a character, prop, 
graphic, etc. Each frame of a video signal may have one or more video objects. The auxiliary 
information is embedded in video or audio signals using "steganographic" methods, such as 
digital watermarks. Further column 3, line 55 through column 4, line 18, the video object exists 
in a "video sequence" for some duration, such as a contiguous set of video frames. A single 
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image instance in a frame corresponding to the object is a video object layer. The video object 
may comprise a sequence of natural images that occupy a portion of each "frame in a video 
sequence (frame to frame)", such as a nearly static talking head or a moving athlete. 
Alternatively, the video object may be a computer generated rendering of a graphical object that 
is layered with other renderings or natural images to form each frame in a video sequence, also 
column 8, lines 19-32, the watermark encoder may encode auxiliary information throughout the 
entire video frame in which at least one marked video object resides. For example, the user may 
specify via the editing tool the location of two or more video objects by drawing a boundary 
around the desired video objects in a video sequence. The encoding process records the screen 
location information for each object in the relevant frames and associates it with the auxiliary 
information provided by the user, such as an object identifier. The encoder then creates a 
watermark message for each frame, including the screen location of an object for that frame and 
its object identifier. Next, it encodes the watermark message repeatedly throughout the frame). 

In response to Applicant's argument regarding claim 1, that Rhoads does not disclose 
"defining a bounding box around the image to provide an area of interest". 

The Examiner disagrees and indicates that Rhoads discloses (column 6, lines 23-29, the 
decoding process, the identifier closest to the location of the user interaction is used. A 
modification includes providing bounding locations in the watermark and determining whether 
the user's selection is within this area, as opposed to using the closest watermark location to the 
user's selection, also column 9, lines 13-26, video object locations and selecting them for 
watermark encoding, a watermark encoding process 516 proceeds to encode an object identifier 
for each selected object. The watermark may be encoded in screen locations and frames occupied 
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by a corresponding video object. Alternatively, object identifiers and corresponding screen 
location information may be encoded throughout the video frames, also column 9, lines 63 
through column 10, line 3, and further lines 20-32, process includes screen locations 606 of the 
objects that may be calculated on screen location co-ordinates, the screen extents may be a 
bounding rectangle, the encoder computes a bounding region for each object (608), if not already 
available. The bounding region of a video object instance refers to a bounding rectangle that 
encompasses the vertical and horizontal screen extents of the instance in a frame. The watermark 
block size refers to a two dimensional screen space in which the watermark corresponding to a 
video object, or set of objects, is embedded in a frame at a given encoding resolution. The 
watermark encoder then proceeds to embed a watermark in non-transparent blocks of the 
bounding region. Finally column 11, lines 16-45,the first process extracts auxiliary information, 
such as object identifiers and screen locations (bounding box rectangle) the fourth renders 
(decodes) watermarked information). 

Claim Rejections - 35 USC § 102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another filed in the United 
States before the invention thereof by the applicant for patent, or on an international application by another who 
has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371(c) of this title before the invention 
thereof by the applicant for patent. 

3. Claims 1-5, are rejected under 35 U.S.C. 102(e) as being anticipated 
by Rhoads et al (U.S. patent 7,050,603). 
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Regarding claim 1 , Rhoads discloses a method comprising, examining the video signal 
being generated by said computer system and applied to a display screen to locate image areas in 
which the video signal changes from frame to frame (column 5, lines 39-53, on the end user side, 
there are two places for network connectivity, rendering of linked information, and user 
interaction. Internet connectivity can be included in the video display device or associated set-top 
box or in a portable display device, such as a personal laptop. The rendering of the linked 
information can occur on the video display, possibly using picture-in-picture technology so 
others can still see the original video, or in the portable display device, such as a laptop since 
Internet browsing can be a personal experience. User interaction with the system, such as 
selecting the object to find linked information can happen with the video display, such as 
pointing with a remote, or with a portable display device, such as using a mouse on a laptop. 
Specific implementations can include a variety of combination of these components, also column 
6, lines 15-21, in other words, a process at the end-user side maps the location of the user 
selection to an identifier based on the locations encoded along with the identifiers in the content. 
For example, a segment of the audio track that is intended to be played with a corresponding 
video "frame to frame" sequence may include a watermark or watermarks that carry one or more 
pairs of identifier and locations); 

defining a bounding box around said image areas to provide an area of interest (column 6, 
lines 25-26, a modification includes providing bounding locations in the watermark); 

and detecting the watermark in said area of interest ( column 6, lines 26-29, determining 
whether the user's selection is within this area, as opposed to using the closest watermark 
location to the user's selection). 
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Regarding claim 2, Rhoads discloses the method of claim 1 , wherein the bounding box is 
rectangular (see claim 1, also column 10, lines 1-2, the screen extents may be as coarse as a 
bounding rectangle or a polygonal shape entered by drawing a boundary around an object via a 
video editing tool). 

Regarding claim 3, Rhoads discloses the method of claim 2, including scaling the area of 
interest to a predetermined resolution (column 11, lines 32-45, a more efficient approach is to 
implement a watermark screen that invokes a watermark decoder only when watermark data is 
likely to be present. A control signal sent in or with the video content can be used to invoke a 
watermark decoder. The control signal may be an in-band signal embedded in the video content, 
such as a video or audio watermark. For example, a watermark detector may look for the 
presence of a watermark, and when detected, initiate a process of decoding a watermark payload, 
accessing information or actions linked via an object identifier in the payload, and displaying the 
linked information or actions to the user. The control signal may be one or more control bits in a 
watermark payload decoded from a watermark signal, also column 13, and lines 47-49, the 
location code is specified at a reference frame resolution, and the user selection coordinates are 
normalized to this reference resolution). 

Regarding claim 4, Rhoads discloses the method of claim 1, including examining the 
video signal for further areas of interest, and detecting the watermark in the further areas of 
interest (column 12, lines 8-17, the decoding process may focus a watermark decoding operation 
on a spatial region around a screen location of a video display selected by the user. Alternatively, 
the user might issue a command to look for enabled content, and the decoding process would 
initiate a watermark detector on frames of video or audio content in temporal proximity to the 
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time of the user's request. The decoding process may buffer frames of the most recently received 
or played audio or video for the purpose of watermark screening in response to such requests). 

Regarding claim 5, Rhoads discloses a computer system comprising, means for examining 
the video signal being generated by said computer system and applied to said display screen to 
locate image areas in which the video signal changes from frame to frame; means for defining a 
bounding box around said image areas to provide an area of interest; and a watermark detector 
for detecting the watermark in said area of interest (column 12, lines 19-25, one configuration is 
video player with an interactive user interface that displays video content and implements 
watermark enabled features. In this configuration, the player decodes the watermark, displays 
video content, and enables the user to select video objects via its interactive user interface. The 
player may have a local database for looking up the related information or action of an identifier 
extracted from a video object, also column 12, line 60 through column 13, line 8, the local 
processing system renders the video content 810. In a PC, the rendering process includes 
converting the video signal to a format compatible with the video controller in the computer and 
writing the video to video memory in the video controller 812. The video controller 812 then 
displays the video signal on a display device 814. As the video is being rendered, the local 
processing system buffers frames (816) of audio or video for watermark detecting and decoding. 
In a PC, the buffering may be integrated with rendering the video to video memory or may be 
implemented as a separate process (e.g., allocating separate video buffers in main memory or 
video memory). Also, depending on the nature of the video signal and encoding process, the 
buffer may store frames of compressed video content or decompressed video content from which 
watermarks are detected and decoded). 
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Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent ma\ not be obtained though the in\ ention is not identically disclosed or described as set forth in section 102 of this title, if the 
differences between the subject mallei sought to be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person ha\ ing ordinary skill in the art to which said subject matter pertains. Patentability 
shall not be negatived by the manner in which the invention was made. 

5. Claims 6-9, are rejected under 35 U.S.C. 103(a) as being unpatentable over Rhoads et al 
(U.S. patent 7,050,603) in view of England et al (U.S. patent 7,203,310). 

Regarding claim 6, Rhoads discloses (FIG. 1A, step 104, column 4, lines 8-30, is a flow 
diagram depicting a process for encoding and decoding watermarks in content to convey 
auxiliary information 100 about video objects in the content. An embedding process 102 encodes 
the auxiliary information into a watermark embedded in the video content. A transmitter 104 
then distributes the content to viewers, via broadcast, electronic file download over a network, 
streaming delivery over a network. A receiver 106 captures the video content and places it in a 
format from which a watermark decoder 108 extracts the auxiliary information. A display 110 
displays the video to a viewer. As the video is being displayed, a user interface 1 14 executes and 
provides visual, audio, or audio-visual information to the user indicating that the video is 
embedded with auxiliary information or actions. This user interface may be implemented by 
superimposing graphical information over the video on the display 110. Alternatively, the 
decoder can pass auxiliary object information to a separate device, which in turn, executes a user 
interface. In either case, the user interface receives input from the user, selecting a video object. 
In response, it performs an action associated with the selected object using the auxiliary object 
information decoded from the watermark). 
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However regarding claim 6, Rhoads does not explicitly state, "a graphics card for use in a 
computer system". On the other hand England in the same field of cryptographically protecting 
secure content teaches (column 11, line 44 through column 12, line 3, secure graphics cards must 
be able to authenticate themselves as such. In particular, trusted software must be able to 
distinguish a secure graphics cards from a traditional graphics cards or a circumvention device, 
such as a spoof. In addition, trusted software must be able to reveal cryptographic keys to the 
graphics cards and be able to verify that the receiver of the keys is indeed a secure graphics 
cards. For this purpose, secure graphics cards are equipped with a crypto processor in accordance 
with the invention, which performs the cryptographic tasks of authentication and key transport. 

Therefore, it would have been obvious to a person of ordinary skill in the art at time the 
invention was made, to modify Rhoads invention according to the teachings of England because 
combination of Rhoads and England provides techniques for crypto-geaphically securing content 
routed through a graphics pipeline, providing both confidentiality and tamper protection with 
respect to the content. 

Regarding claim 8, Rhoads discloses a computer system comprising: a display engine that is 
configured to generate a video signal that includes one or more display windows (see claim 6, 
also column 5, lines 39-53, on the end user side, there are two places for network connectivity, 
rendering of linked information, and user interaction. Internet connectivity can be included in the 
video display device or associated set-top box or in a portable display device, such as a personal 
laptop. The rendering of the linked information can occur on the video display, possibly using 
picture-in-picture technology so others can still see the original video, or in the portable display 
device, such as a laptop since Internet browsing can be a personal experience. User interaction 
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with the system, such as selecting the object to find linked information can happen with the 
video display, such as pointing with a remote, or with a portable display device, such as using a 
mouse on a laptop. Specific implementations can include a variety of combination of these 
components); 

a detector that is configured to identify locations of changes of picture element 
values in the video signal, a processor that is configured to define a bounding box based on the 
locations of changes (Column 1, lines 41-68) a media signal, such as an image (video)signal, is 
modified such that the embedded code (watermark) is imperceptible or nearly imperceptible to 
the eye, yet may be detected (located) through an automated detection process, the reading 
component analyses content (examines) content to detect whether a watermark is present, video 
objects in a video sequence (frame-to-frame) with object specific actions or information 
(watermark), a video object refers to a spatial or temporal portion of a video signal. Also, 
(Column 4, lines 16-18, captures video content and places it in a format from which a 
watermark decoder 108 extracts the auxiliary information .A display displays the information to 
a viewer. (Column 9, lines 63 - Column 10, line 3 ) process includes screen locations 606 of the 
objects that may be calculated on screen location co-ordinates, the screen extents may be a 
bounding rectangle, (column 11, lines 16-45),the first process extracts auxiliary information, 
such as object identifiers and screen locations (bounding box rectangle) the fourth renders 
(decodes) watermarked information. 

Regarding claim 9, Rhoads discloses the computer system of claim 8, including a 
resolution converter that is configured to scale at least a portion of the video signal (column 11, 
lines 32-45, a more efficient approach is to implement a watermark screen that invokes a 
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watermark decoder only when watermark data is likely to be present. A control signal sent in or 
with the video content can be used to invoke a watermark decoder. The control signal may be an 
in-band signal embedded in the video content, such as a video or audio watermark. For example, 
a watermark detector may look for the presence of a watermark, and when detected, initiate a 
process of decoding a watermark payload, accessing information or actions linked via an object 
identifier in the payload, and displaying the linked information or actions to the user. The control 
signal may be one or more control bits in a watermark payload decoded from a watermark signal, 
also column 13, and lines 47-49, the location code is specified at a reference frame resolution, 
and the user selection coordinates are normalized to this reference resolution, also column 6, 
lines 15-21, in other words, a process at the end-user side maps the location of the user selection 
to an identifier based on the locations encoded along with the identifiers in the content. For 
example, a segment of the audio track that is intended to be played with a corresponding video 
"frame to frame" sequence may include a watermark or watermarks that carry one or more pairs 
of identifier and locations). 

With regard to claim 7 the arguments analogous to those presented above for claims 1 
and 6 are respectively applicable to claim 7. 

6. Claims 10 and 13-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Rhoads et al (U.S. patent 7,050,603) in view of England et al (U.S. patent 7,203,310) as applied 
to claims above and further in view of Tinker et al (U.S. 6, 829,301). 

However regarding claim 10, Rhoads does not explicitly state "RGB-to-Y converter". On 
the other hand Tinker, teaches (column 14, lines 43-55, the format converter 502 converts an 
input RGB video signal SIR, S1G and SIB into a full depth luminance signal Y, a full depth in- 
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phase chrominance signal and a full depth quadrature-phase chrominance signal Q'. The full 
depth luminance signal Yis coupled to a luminance input Y of first MPEG encoder 520. The full 
depth in-phase chrominance signall' and full depth quadrature-phase chrominance signal Q' are 
coupled to, respectively, first LL spatial filter 504 and second LL spatial filter 506. The full 
depth quadrature -phase chrominance signal Q' is also coupled to LH spatial filter 508 and HL 
spatial filter 510. The full depth in-phase chrominance signal I' is also coupled to a luminance 
input Y of the second MPEG decoder 522. 

Therefore it would have been obvious to a person of ordinary skill in the art at time the 
invention was made, to modify Rhoads and England invention according to the teachings of 
Tinker because its provides enhanced information quality and security, which easily can be 
implemented in an imaging device such video camera. 

With regard to claims 13-20 the arguments analogous to those presented above for 
claims 1, 2, 3, 4, 5, 6 and 10 are respectively applicable to claims 13-20. 

Allowable Subject Matter 

7. Claims 1 1 and 12 objected to as being dependent upon a rejected base claim, but would 
be allowable if rewritten in independent form including all of the limitations of the base claim 
and any intervening claims. 

Conclusion 

8. THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of 
the extension of time policy as set forth in 37 CFR 1.136(a). 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Contact Information 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Seyed Azarian whose telephone number is (571) 272-7443. The 
examiner can normally be reached on Monday through Thursday from 6:00 a.m. to 7:30 p.m. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehta Bhavesh, can be reached at (571) 272-7453. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application information Retrieval (PAIR) system. Status information for published application 
may be obtained from either Private PAIR or Public PAIR. 

Status information about the PAIR system, see http:// pair-direct.uspto.gov. Should you have 
questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 
866-217-9197 (toll-free). 

/Seyed Azarian/ 

Primary Examiner, Art Unit 2624 
Group Art Unit 2624 
October 5, 2008 
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